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Probabilistic automata (p.a.) are a generalization of finite deter- 
ministic automata. We follow the formulation of finite automata in 
Rabin and Scott (1959) where the automata A have two-valued out- 
put and thus can be viewed as defining the set T(N) of all tapes ac- 
cepted by X. This involves no loss of generality. A p.a. is an automa- 
ton which, when in state s and when input is o, has a probability 
pils, c) of going into any state s;. With any cut-point 0 <A < 1, 
there is associated the set T(N, X) of tapes accepted by A with 
cut-point i. 

Here we develop a general theory of p.a. and solve some of the 
basic problems. Aside from the mathematical interest in pursuing 
this natural generalization of finite automata, the results also bear 
on questions of reliability of sequential circuits. 

P.a. are, in general, stronger than deterministic automata 
(Theorem 2). By studying the way we may want to use p.a. we are led 
to introduce the concept of isolated cut-point. It turns out that every 
p.a. with isolated cut-point is equivalent to a suitable deterministic 
automaton (the Reduction Theorem 3). It is interesting to note 
that in passing from a minimal deterministic automaton to 
an equivalent p.a. we can sometimes save states (Section VII). 

The Reduction Theorem is applied to prove the existence of an 
approximate calculation procedure for a calculation problem in- 
volving products of stochastic matrices (Section VIII). The problem 
is of a new kind in that there is no a-priori bound on the number of 
operations (matrix multiplications) which we may have to perform 
and therefore classical numerical estimates of round-off errors do 
not apply. 

Actual automata (Definition 9) have the property, often existing in 


* Most of the results in this paper were established while the author was visiting 
Bell Telephone Laboratories, Murray-Hill, in the summer of 1960. This paper was 
presented as an invited address at the Annual SIAM Meeting at California In- 
stitute of Technology, March 1962. 
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actual unreliable circuits, that all transition probabilities are strictly 
positive. Actual automata are proved to give only definite events. 
This points to the restrictions we may have to impose on a 
probabilistic sequential circuit if we want it to perform general tasks, 
namely, some transitions should be prohibited. 

Finally we treat the important problem of stability. Is the opera- 
tion of a p.a. stable (unchanged) under small enough perturbations 
of the transition probabilities? We have an affirmative answer to this 
question in the case of actual automata (Theorem 11) and we discuss 
the problem for the general case. 


INTRODUCTION 


Finite automata are mathematical models for systems capable of a 
finite number of states which admit at discrete time intervals certain 
inputs (incoming signals) and emit certain outputs. If the system is in 
state s and the input is e then the system will move into a new state s; 
which depends only on s and c and will have an output which depends 
only (is a function of) on s;. Thus the system will transform a sequence 
of inputs into a sequence of outputs and the relevant aspect of the 
system is this transformation. Sequential circuits, and even whole 
digital computers, provided the computer operates using only internal 
memory or just a fixed amount of tape, are systems which behave like 
finite automata. There is an extensive literature on finite automata. In 
this paper we follow the notations and use some of the results on autom- 
ata contained in the paper by Rabin and Scott (1959). In particular 
the formulation given there amounts to assuming that the set of outputs 
contains just two elements. This is a convenient restriction which we 
follow also here but the results immediately extend to the general case of 
more than two outputs. Because of the restriction to two-valued outputs 
automata can be viewed as defining sets of sequences of inputs (tapes) 
and this point of view is adopted throughout this paper. 

Finite automata exhibit a deterministic behavior. The state s and 
input o determine the next state of the automaton. It is quite natural to 
consider automata with stochastic behavior. The idea is that the au- 
tomaton, when in state s and when the input is c, can move into any state 
si and the probability for moving into state s; is a function p;(s, c) of 
sand øe. 

A practical motivation for considering probabilistic automata is that 
even the sequential circuits which are intended to be deterministic 
exhibit stochastic behavior because of random malfunctioning of com- 
ponents. This situation was first taken up by von Neumann (1956) 
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who considered schemes for organizing combinatorial (and to some 
extent also sequential) circuits constructed with specific components so 
as to increase their reliability. 

Though the generalization from the abstract deterministic automata 
to the abstract probabilistic automata (p.a.) lies near at hand, there are 
no general results about p.a. in the literature. In particular, it was not 
even known whether p.a. can do more than deterministic automata. In 
this paper we develop a general theory of p.a. and answer some of the 
basic questions about them. 

It turns out that in general p.a. are stronger than deterministic 
automata. We introduce, however, a new concept of isolated cut-point 
and prove the fundamental Reduction Theorem 3 that every p.a. with 
isolated cut-point is equivalent to a suitable deterministic automaton. 

In Section XI we define actual automata which are automata such that 
all their transition probabilities are strictly positive. These automata 
define a very limited class of regular events (Theorem 10). The results 
are of some significance for the theory of reliability. They indicate that 
if we want to synthesize general sequential circuits from unreliable 
components we must organize them so that transitions between certain 
states are prohibited (have probability zero), or else consider the circuit 
as having broken down if these transitions occurred. 

Another problem bearing on theory of reliability is the stability prob- 
lem. The probabilistic automaton is called stable if its behavior is not 
changed by small perturbations of the transition probabilities. In 
synthesizing circuits from unreliable components we surely want to get 
stable circuits. In Section XII we give a stability theorem for actual 
automata. We also discuss the general stability problem but leave it 
open. 


I. FINITE AUTOMATA 


In this section we give a brief resume of the basic definitions and some 
basic results which will be used in the sequel, from the theory of finite 
(deterministic) automata. The exposition follows closely that in Rabin 
and Scott (1959). By “automaton” we shall mean, throughout this 
section, deterministic automaton. 

Let È be a finite nonempty set, to be called the alphabet. Letters r, o 
(with subscripts) will usually denote elements of Z. The set of all finite 
sequences of elements of 2 will be denoted by =*. The elements of =* 
will be called tapes. The letters z, y, z, u, v (with subscripts) will always 
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denote tapes. The empty tape (i.e., the sequence of length zero) will be 
denoted by A. Subsets of =* (i.e., sets of tapes) will sometimes be called 
events. 

If x = o ++- ois a tape then the length I(x) of x is (x) = k. If z 
and y are tapes then zy will denote the concatenation of x and y. Note 
that =* with this operation zy is a free semi-group with the elements of 
Z as free generators. 

DEFINITION 1. A finite (deterministic) automaton over = is a system 
X = (S, M, so, F) where S is a finite set (the set of states), M is a func- 
tion from S X Z into S (the transition table), so € S (the initial state), 
and F C S (the set of designated final states). 

M can be extended to a function from S X =* to S by, M(s, A) = s, 
M(s, xo) = M(M(s, x), c) (s € S, £ € 2*,0 € 3). M(s, x) is the 
state in which Y “gets off” the tape z if it started on z in state s. 

DEFINITION 2. A tape x is said to be accepted by N if and only if 
M (so, x) € F. The set defined by is the set of all tapes accepted by 
N, and is denoted by T(A). An event U C &* is called a regular event if 
for some finite automaton A, U = T(Y). 

Every finite event is regular. If U and V are regular so are U N V, 
UU Y and =* — U (see Rabin and Scott, 1959). 

In Rabin and Scott (1959) a necessary and sufficient condition for an 
event T C Z” to be regular was given in terms of right equivalence 
relations. 

DeErINITION 3. Let T C 2*, the right-equivalence relation = p generated 
by T is defined as follows. For z, y € 3", x =r y if and only if for all 
z € =” we have: xz € T if and only if yz € T. 

It is easy to see that =, is an equivalence relation on 2*. Note that 
= is right-invariant with respect to the multiplication of the semigroup 
x", i.e., for all z, y,2 € D“, if z =r y then zz =r yz. 

TurorEM 1 (Rabin and Scott, 1959). A set T C X* is a regular event 
if and only if the number of equivalence classes of =* by the equivalence 
relation =r ts finite. If the number of equivalence classes ise < œ then 
for a suitable X, T = T(N) where the automaton A hase states. No automa- 
ton with fewer than e states defines T. 


H. PROBABILISTIC AUTOMATA 


We shall now define the basic concept of this investigation, namely 
the concept of probabilistic automata. It will be seen that probabilistic 
automata are like the usual automata except that now the transition 
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table M assigns to each pair (s, o) € S X Z certain transition probabili- 
ties. 

DEFINITION 4. A probabilistic automaton (p.a.) over the alphabet 2 
isa system A = (S, M, so , F} where S = {s,..., Sn} is a finite set 
(the set of states), M is a function from S X E into [0, 1]"™ 1 (the 
transition probabtlitves table) such that for (s,s) € SX È 


M(s, o) = (pols, e), AS Prl8, o)), 
0s pis, o), 2 pils; o) = L, 


so € S (the initial state), and F C S (the set of designated final states). 

Probabilistic automata are models for systems (such as sequential 
circuits) capable of a finite number of states so, ..., Sa. The system 
may receive inputs e € Z. When in state s and if the input is o then the 
system can go into any one of the states s; € S and the probability of 
going into s; is the (¢ + 1)th coordinate p;(s, c) of M(s, c). These 
transition probabilities p;(s, c) are assumed to remain fixed and be 
independent of time and previous inputs. Thus the system also has defi- 
nite transition probabilities for going from state s to state s; by a se- 
quence x € &* of inputs. These probabilities are calculated by means of 
products of certain stochastic matrices which we shall now define. 

Derinttion 5. For o € Z and x = ojo. ->> om define the n + 1 by 
n + 1 matrices A(c) and A(x) by 


A(c) = [p;(s:, o)lo<e<nosi<n 
A(x) = A(oi)A(o2) +++ Alom) = [pi(si, x) losisnosisn- 
Remark. An easy calculation (involving induction on m) will show 
the (i + 1,7 + 1) element p;(s:, x) is the probability of Y for moving 
from state s; to state s; by the input sequence x. 


Derrtnition 6. If Y = (S, M, s, F) and F = {s,,, ..., s:$, 7 = 
fio, .-., tt, define 


p(z) = 25 pi(so, 2). 
ter 
p(x) clearly is the probability for N, when started in so , to enter into a 
state which is member of F by the input sequence v. 


1 (0, 1] is the closed unit interval 0 < x < 1. [0, 1]**1 is the set of all n + 1-tuples 
(zo, -.. ,%n) where 0 Sa; S1. 
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Ill. SETS OF TAPES DEFINED BY P.A. 


A p.a. X may be used to define sets of tapes in a manner similar to 
that of deterministic automata except that now the set of tapes will 
depend not just on Ñ but also on a parameter À. 

DEFINITION 7. Let N be p.a. and à be a real number, 0 S A < 1. 
The set of tapes T(N, A) is defined by 


T(H, A) = fajx € BA < p(a)}. 


If x € T(N, A) we say that x is accepted by Ñ with cut-point X. T( YM, X) 
will also be called the set defined by N with cut-point i. 

REMARK. Deterministic automata can be considered as a special case 
of p.a. Namely, if in Definition 1 M(s, e) = s; then we can view this 
as if 2 will enter state s; with probability 1. Thus in rewriting the deter- 
ministic automaton as a p.a. the stochastic vectors M (s, co) = (po, ...; 
Pn) Will have exactly one coordinate 1 and all the others 0. It is readily 
seen that in this case p(z) = 1, for æ € &", if and only if x € T(). 
Hence for any A, 0 < à < 1, we have T(N) = T(M, X). Thus every set 
definable by a deterministic automaton is trivially definable by some 
p.a. In the next section we shall see the converse is not true and that 
therefore p.a. give a strictly larger class of definable sets. 


IV. PROBABILISTIC AUTOMATON DEFINING NONREGULAR EVENT 
The folowing matrices were suggested by E. F. Moore. 


_fi o ae 
r=) Bs 1 


It can be readily verified that if 


m 
Pat Paz: Da P=[" a ô; € {0, 1} 
then p = + nôn -+ 6: where p is written in binary expansion. 


THEOREM 2. Let A = (S, M, so, F) be an automaton over È = {0, 1} 
such that S = {s,s}, A(0) = Po, A(1) = Pi, F = {sı}. There exists a 
0 < A < 1 such that T(Y, X) is not definable by a deterministic automaton 
(is not a regular event). 

Proor: If x = ôð --- 6 € =”, then by the above, p(z) = 
. nôn e by. 

The values p(z) are dense in the whole interval [0, 1]. This implies 
that fO0 S A < M < 1 then T(M, M) C TM, AX) where the inclusion is 
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proper. The sets T(N, A), 0 < à < 1, therefore form a nondenumerable 
pairwise different collection of sets. But there is only a denumerable 
collection of regular events. Therefore there exists a à such that T(N, A) 
is not regular. 

Remark. The above argument is a pure existence proof. We can, 
however, present a specific \ such that T(N, X) is not regular. Namely, 
let w, , w2, ..., be any enumeration of =* then for \ = ww «++, T( A, A) 
is not regular; we omit the proof. 

The above J is irrational. It can in fact be shown that if A is rational 
then T(N, A) is a regular event. 


V. ISOLATED CUT-POINTS 


Let % be a p.a. and 0 < à < 1. Given a tape x € &* we devise the 
following probabilistic experiment EF to test whether x € T(M, X). 
We run z through Y a large number N of times and count the number 
m(E) of times that Y ended in a state in F. If X < m(E)/N we accept 
x and otherwise we reject it. Because of the probabilistic nature of the 
experiment it is of course possible that we sometimes accept x even 
though z € T(N, X) or reject it even though z € T(N, A). By the law 
of large numbers, however, there exist for each x such that p(x) = ` 
and each 0 < ea number N (z, e) such that 

Pr(zIa E e Woy one T(t, )) >l—e 
That is, the probability of obtaining the correct answer by the experi- 
ment E (consisting of running x N (z, e) times through Y and counting 
successes) is greater than 1 — e. 

To perform the above stochastic experiment we must know N(z, e) 
which depends on | p(x) — |. Thus we have actually to know p(x) 
in advance if we want to ascertain whether x € T(N, >) with probability 
greater than 1 — e of being correct. Once we know p(x), however, the 
whole experiment E is superfluous. 

The way out is to consider values à such that | p(x) — A | is bounded 
from below for all z € =*. 

Derinition 8. A cut-point à is called isolated with respect to % if 
there exists a 0 < 6 such that 


8 < |p(z) — à| forall z € 2%. (1) 
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Remark. It is readily seen that there exists an integral valued func- 
tion N (ô, e) such that for an isolated à and any z € 3* 


mE) 
N (6, €) 
Thus the proposed stochastic experiment for determining whether 


x € T(Y, A) can be performed without any a-priori knowledge of p(x). 
This fact makes it natural to consider isolated cut-points. 


Pr(E Ja < ose TAN) B16 


VI. THE REDUCTION THEOREM 


THEOREM 3. Let AY be a probabilistic automaton and ` be an isolated 
cut-point satisfying (1). Then there exists a deterministic automaton B 
such that T(%,) = T(%). If A has n states and F consists of just one 
state then Y can be chosen to have e states where 


e < [1 + (1/8)? (2) 


Proor: Let the set of states S be fso, -.., Sn} and F = {sna}. For 
every tape x, A(x) isann X n matrix and p(x) is the upper left corner 
element of A(z). 

Let z1, ..., 2% be tapes which are pairwise inequivalent by = ran 
(ef. Definition 3). Thus for every 7 < k, j < k, i Æ j, there exists a 
tape y such that 


ey E T(A, à), ay € T(A, A) (3) 


or vice-versa. Let the first row of A(x:),1 S i S n, be (5, «++, En’) 
and the last column of A (y), for the particular y appearing above, be 
(m, °+*, Mma). From A(zy) = A(z:)A(y) and A(xy) = A(z;)A(y) 
it follows that 


pley) =i mt + Em, play) = imt Him. 
Combining with (3) we get 
N< alm te tims amtoe + be’ SA. (4) 
Since A is isolated and 6 < | p(x) — A | for x € B*, (4) implies 
28 S (4° am + ++ + Gn’ — be’) mn. (5) 


2 If F contains r states then the bound is e < (1 + (r/8))*"! and the proof is es- 
sentially the same. 
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Taking absolute values and observing that the 7;, as elements of a 
stochastic matrix, satisfy | 7; | < 1, (5) leads to 


S aJa a l +--+ lit Ent) for ij. (6) 


An argument involving volumes in n-dimensional space will now be 
used to infer from (6) a bound on k. The n-tuples (1, ---, n) will be 
considered as points of Euclidean n-space. Let c;, 1 < i < k be the set 


= fh, o E L u, L Sj Sm 01 (& — &') = 3} 
I 
Each o; is a translate of the set 


= {(4,°',&) {0 &, 1 Sj Sn, Db = 3. 


The set o is readily seen to be an (n — 1)-dimensional simplex which 
is a subset of the hyperplane zı + --- + £a = 6. Then — 1 dimensional 
volume V,_:(c) of o, expressed as a function of 4, is ci” where c is 
some constant not depending on 6. 

From >); &;' = 1 it follows that (&, =+, En) € o; mee 


De = 1 +4, = &;,1 <j sn 
J 


Thus o; GC r where 


={@, 5 &) [De =1+3408h,1 85 5 xn). 


Figure 1 shows the sets o;, 1 S 7 < k, and 7, for the two-dimensional 
casen = 2 and for k = 3. The point wi coordinates (£, £°) is denoted 
by P;, 1 S183. 


Fig. 1. The sets c; and r for n = 2 and k = 3 
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A point ($i, +--+, &) € c; is an interior point of c; (in the topology 
of the hyperplane zı + --- + a = 1 + ô) if and only if 0 < £p — Ep 
for 1 S p < n. Because of (6) c; and o; ,7 # 7, have no interior points 
in common. For otherwise, if (&, ---, a) is interior to both e; and 
c; , we would have 0 < £p — &',0 < £p — £p and hence 


| Ep — &’ | < lés — & | + | & éh lsSpsn. 


Hence 
Dli — &' |< le t lt le E | =54+4, 


contradicting (6). 
Thus for 7 Æ j, o; and ø; have no interior points in common. This 
implies 


heb" = Vaala) + + Valor) S Vea(r) = oll + 8) 


Hence k < [1 + (1/8)]””. Thus the number of e equivalence classes 
of the relation = ra, is at most [1 + (1/5)]”*. By Theorem 1, 7(%, X) 
is definable by an automaton $ with e states. 


VII. SAVING OF STATES 


From the proof of the Reduction Theorem 3 and the estimate (2) 
given there, it seems possible that in passing from a p.a. Y to an equiv- 
alent deterministic automaton we may have to increase the number of 
states. In other words, the p.a. is more economical in terms of number of 
states. The following theorem shows that this does in fact happen in 
certain cases. 

THEOREM 4. There exists an automaton A with just two states and a 
sequence Mn, 1 Sn < ©, of isolated cut-poinis such that for each n, the 
automaton Y, with the least number of states which satisfies T(N, dn) = 
T(8,) has at least n states. 

Proor: Let 2 = {0, 2}, S = {s0, s}, and F = {sı}. Let the transi- 
tion probabilities be such that 


ww- J o-i fh 


It is easy to verify that if z = 8d) --- ôn € 2* then 


dy 
371 à 


Ôn , Ôn 
d e Fee 
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Remembering that 6; € {0, 2} we see that the topological closure C of 
the set P = {p(x) | x € =*} is precisely Cantor’s discontinuum. 

Thus all the points 4, 0 S à < 1, which satisfy à € C are isolated 
cut-points for N. Consider now the real number (written in ternary 
notation) A, = + 22 --- 211 where the number of digits is n + 1. For 
x € X” to satisfy àn < p(z) it is necessary and sufficient that z have the 
form x = 2,22 --- 2 where zı € =* and the number of 2’s is at least n. 
Thus the set T(N, An) is nonempty and if x € T(N, An) then n < I(x). 
It follows from elementary theory of automata (see Rabin and Scott, 
1959, Theorem 7) that the minimal deterministic automata %, for 
which T(N, A») = T(%,) has at least n + 1 states (in fact, %, has 
exactly n + 1 states). 

Remark. An analysis of the possible values of p(x) will show that 
for the i, as above 


. . 1 
Jim inf | p(x) ZM] = gone 


Thus in (2) we can take ô = 3°” * which gives for the number of states 
the bound 1 + 3”*™. In this case the bound turns out to be much too 
large. We do not know whether in other examples the bound is sharper 
or whether the bound in Theorem 3 can in fact be greatly improved. 


VIII. APPROXIMATE CALCULATION OF MATRIX PRODUCTS 


Let H be a finite set H = {P,, ---, Pa} of stochastic n X n matrices 
and let 0 < ebe a given real number. The elements of all matrices P € H 
are assumed to be finite decimal fractions.’ Consider the following com- 


putational task. At discrete time intervals m = 1, 2, ---, we are pre- 
sented with matrices Pa € H, Pa € H, --+. Let £m = tig +++ in, 
m = 1,2, ---. After each time m we wish to know, within e, the element 


Pon (1, n) of the product 
Hem = [Pen(ts Jlisesnisisn = PaPa +++ Pin 


of the matrices given thus far. Since we are thinking here in terms of 
actual calculation (using, say, an actual computer with a fixed memory) 
it is not possible, in general, to solve our problem by calculating Msn , 
at all times m, with complete accuracy. The elements p,,,(¢, J) will 
have more and more decimals and recording and calculating with these 


3 This restriction on the matrices P € H is not essential and is included just 
in order that we can say that the matrices P € H are actually “given.” 
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numbers will become impossible with increasing m. Nor is it possible to 
adopt a simple rounding-off procedure because the number m of matrix 
multiplications that we have to perform is not bounded in advance. 
Thus a fixed rounding-off procedure may result in a cumulative error 
which will become larger than e. 

We shall apply our reduction theorem (Theorem 3) to show that 
under certain conditions on H this problem of approximate calculation 
can be actually solved. The solution rests on the following: 

THEOREM 5. Let = = {1, ---, k} and for x = iis +--+ im € 2” let pz 
denote the (1, n) element of the product P;,P:, +--+ Pin- Assume that H 
is such that V = {pa x£ € =} is nowhere dense in the interval {0, 1]. 
Then for every 0 < e there exists an integer h, real numbers M, +++, An; 
and deterministic automata %, , ---, Ar, over È such that 


G)0O=rA <e< Ae < M= lA - MM GIL Sth. 
(ii) Ss Pez <í Aiti iff x € T(A;) = T (Miss), 1 < t <h. 


Proor: V nowhere dense means that the topological closure V does 
not contain any nontrivial interval. Thus there exists, for some integer 
h, a sequence \;,1 S i S h, satisfying (i) and also à; € Vfor2 Sis 
h- 1. 

Consider the p.a. A over È having the states so, --+, Sn, the set 
{sna} of designated final states, and transition probabilities such that 
the matrix corresponding to í € X is P;. We have for x € 3”, p(x) = 
Pas . The numbers A;,2 S 7 S k — 1 are isolated cut-points for Y. Thus 
by Theorem 3 the set 7'(2, \.), 2 S i < h — 1, is definable by some 
deterministic automaton 7'(%;). Hence M; < Ps for2 Si Shil 
(pz = Xz is not possible since à; is isolated) iff x € T(%;). Let % and 
I, be automata such that 7(%,) = =* and T(%,) = ¢. The automata 
W;,1 S i S h satisfy (ii). 

REMARK. The condition concerning V is satisfied, for example, by the 
set H = {A (0), A(2)} of 2 X 2 matrices defined in Theorem 4. In this 
case V is Cantor’s discontinuum. We do not have, however, a criterion 
for deciding whether a given H satisfies the condition. 

The method for approximate calculation of p,,,(1, n) in the case that 
H satisfies the condition of Theorem 5 is now as follows. Given 0 < e, 
let A, U;, 1 S i S h, satisfy the conditions (i) and (ii) of Theorem 5. 
Using just a fixed amount of computer memory it is possible to simulate 
the automata ;, 1 < i S h. As the matrices Pa, , Pa , +++, are given, 


242 RABIN 


the indexes ú , 72, ---, are fed into the simulated automata. At each 
time instant m the computer checks which automata W; accepted £m = 
ty, +++ im. There exists precisely one jm such that £m € T(U;,,) — 
TAn). For Ajn we have | pz,,(1, Nn) — Aj, | < € and we take };,, as 
the approximation for p,,,(1, 2). 

Thus we have proved the existence of an approximate calculation 
procedure which can actually be carried out by a computer. We do not 
know of a classical numerical-analysis method for obtaining this result. 
In fact, an example due to R. E. Stearns shows that without assump- 
tions on H, a computational procedure need not exist. 


IX. ACTUAL AUTOMATA 


In certain actual situations it is natural to assume about an automa- 
ton % that all transitions between states have strictly positive (though 
sometimes very small) probabilities. This motivates the following defini- 
tion. 

DEFINITION 9. A p.a, A is called an actual automaton if for all s € S, 
si € S, and o € È the transition probability p,(s, c) of moving from 
state s to state s; under input ø satisfies 0 < p;(s, o). 


X. PRODUCTS OF POSITIVE STOCHASTIC MATRICES 


It turns out that actual automata have very special properties. To 
study them we need some results about products of strictly positive 
stochastic matrices. The following Lemma 6 is a restatement, in our 
notation, of Theorem 4.1.3 of Kemeny and Snell (1960); the proof is 
included for the sake of completeness. Corollary 7 and Lemma 8 are 
closely related to Theorems 4.1.4—4.1.6 of Kemeny and Snell (1960) 
except that we treat products of several matrices instead of powers of a 
single matrix. The possibility of this generalization was pointed out by 
Mr. A. Paz. 


Dertnition 10. If a = [aih<ic, is a column vector then || @ || is 
defined as || a || = max; a; — min,;a;.If A isann X n matrix having 
columns a, ---, @ then || A || is defined by || A || = max; || a, ll. 


Lemma 6. If P = [pijh<ij<n isa stochastic matrix and A = mini,; Pij 
and if a = [dih<:<n ts a column vector then 


|| Pe || £ (1 — 2A) jie]. 


Proor: Let Pa = [bi<ien. We may assume, without loss of gen- 
erality, that bı = max;b;, b2 = min;b;,a, = MAX: Q; , and ag = min;a;. 
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We have 


b = Put + Piste +-+ Pinan S put + P22 + Piz S A Ht Pind 
=> Pr2( ay = ay). 


Similarly, replacing in the sum for be the a by az, be 2 a+ 
Pala = Qe). Thus li Pa i = b == bs < (a, a2) (1 Pr2 Pa). But 


a, — a = || a || and, since A < py and A S pa, we have 1 — p — 
Pa S 1 — 2A. This establishes the lemma. 
COROLLARY 7. If H = {Pı , -+> , Pa} where the matrices P;,1 Sisk 


are stochastic and all the elements of the P; are greater than 0 < A 
then for any 1 S ù, --- ,îm Sk, 


| PaPa Ranae Piz |l £ (1 = 2A)", 


Proor: The column vectors œr, °-- ,@n of Pi, satisfy || a; || S 1. 
By Lemma 6, the columns ĝi, +- , Bm of Pim Pin satisfy || 6; || < 
1 — 2A. Repeating this argument m — 1 times we get the result. 


For any m X n matrix A = [a,;] we define |A| = max;,; |a;,|. This 
|A| clearly has the usual properties of a norm. 

Lemma 8. If P is a stochastic n X n matrix and a = [aji<ien is a 
column vector then 


Pa — a| £ || a ||. 


Proor: Let (pı, +--+ , Pa) be the first row of P and let b, be the first 
element of Pa. Then 


|b — al = [pıtı + Spi + Pran — a] S P |a — a| + are 
+ Pa |@n — a] S || |l. 
The same applies to all the other elements of Pa. 
COROLLARY 9. If P ts a stochastic n X n matriz and A is ann Xn 
matrix then |PA — A| < || A ||. 
XI. ACTUAL AUTOMATA AND DEFINITE EVENTS 


It will turn out that the sets accepted by actual automata are just 
those described in the following. 

DEFINITION 11. A set T C =” is called a definite event if for some 
integer k the following holds. If k < I(x) then z € T if and only if 
x = yz where k = l(z) and z € T. 

In (Perles, Rabin, and Shamir, 1963) the properties of definite sets 
and the (deterministic) automata defining them are studied in detail. 
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THEOREM 10. If AN is an actual automaton and » is an isolated cut-point 
then T(N, A) is a definite set. Conversely, every definite set is definable by 
some actual automaton with isolated cut-point. 

Proor: Let 0 < ô < |p(x) — A| and assume that all elements of the 
stochastic matrices A(s), c € = (see Definition 5) are greater than 
0 < A. We assume that NÑ has just one designated final state, say sp_. 
The proof is essentially the same for the general case. 

Let k be such that (1 — 2A)" < 28. For any z = cı --: op € D* of 
length k the matrix A(z) equals A (s1) --- A(s) and thus, by Corollary 
7, satisfies || A(z) || < (1 — 2A)** < 28. 

Since p(x), for x € &*, is the (1,n) element of A(x) we have, by 
Corollary 9, |p(ye) ~ p(2)] < |A(y)A(z) — A(e)| < || AG) ||. Thus 
for z satisfying l(z) = k, |p(yz) — p(z)| < 26. Hence yz € T(N, A) if 
and only if z € T(N, X) which proves that T(N, `) is definite. 

The converse is proved by explicit construction of the actual auto- 
maton defining the definite set T. We leave out the details. 


XII. THE STABILITY PROBLEM 


Consider a p.a. X and an isolated cut-point à. It is natural to ask 
whether the set T(N, X) remains unchanged (stable) under small per- 
turbations of the transition probabilities of Y. Results along this line we 
shall call stability theorems. 

THEOREM 11. Let X = (S, M, so, F) be an actual automaton and A be 
A isolated cut-point. There exists an 0 < e such that for every automaton 

= (S, M’, so, F} with transition probabilities differing from those of 
: by less fies e, \ is an isolated cut-point of W and T(N, A) = T(M',r). 

Proor: Let Alo) and A’ (c) be the matrices corresponding to « € = 
in the automata X and N’ respectively. Let A be the smallest element in 
the A(c) and A’ be the smallest element in the matrices A’(o). We shall 
show that for every 0 < 6, we can find an 0 < e so that 


|A (o) — A"(o)| < 6, E2 (7) 
implies for all z = o1 +++ om 
|A (2) — A’(x)| = |A (o1) +++ Alom) — A (01) +++ A’(om)| <è (8) 


This, of course, implies the theorem. 
Let k be such that (1 — 2A)" < 6,/3. We can choose 0 < e small 
enough so that (7) will imply (a) (1 — 2A')*" < 6,/3, (this can be 
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done because A = lim..A’); (b) for all z such that l(z) < k, |A(z) — 
A'(2)| < 6/3. 

If I(x) < k then (8) holds trivially because of (b). If k < I(x) then 
x = yz where l(z) = k. The matrix A(z) is a product of k of the matrices 
A(c) and therefore, by Corollary 7, || A(z) || S (1 — 2A)*" < 6,/3. 
Similarly, using (a), we have || A’(z) || < 6/3. Now 


|A(z) — A’ (x)| £ |A(y) A(z) — A(z)| 
+ |A"(y)A"(z) — A"(2)| + JA(2) — A(z). 


By Corollary 9 and (b) each of the summands on the right is less than 
61 / 3. 

Remark. Theorem 11 could not hold in full generality for arbitrary 
p.a. with isolated cut-point. For assume that X is a p.a. with isolated 
cut-point à such that T(N, A) is not a definite event. An automaton A’ 
may satisfy (7) and yet have only strictly positive transition prob- 
abilities. In this case either A is not an isolated cut-point of W, or T(N, A) 
is a definite event and hence T(W, A) = T(, X). 

Thus a proposed formulation of a conjectured general stability theorem 
would be: If Y is a p.a. and A is isolated cut-point then there exists an 
0 < e such that for every automaton Y’ with conditions as in Theorem 
11 and such that the matrices A’(o) have zeros where A (e) had zeros, X 
is an isolated cut-point of W and T(W, A) = T(M, X). H. Kesten con- 
structed a neat counterexample to this conjecture. Thus the problem of 
giving suitable extensions of Theorem 11 is completely open. 


RECEIVED: April 12, 1963 
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